Using audit and feedback to guide tailored implementations of measurement-based care in community mental health: a multiple case study.

BACKGROUND: Audit and feedback (A&F) is an implementation strategy that can facilitate implementation tailoring by identifying gaps between desired and actual clinical care. While there are several theory-based propositions on which A&F components lead to better implementation outcomes, many have not been empirically investigated, and there is limited guidance for stakeholders when applying A&F in practice. The current study aims to illustrate A&F procedures in six community mental health clinics, with an emphasis on reporting A&F components that are relevant to theories of how feedback elicits behavior change. METHODS: Six clinics from a larger trial using a tailored approach to implement measurement-based care (MBC) were analyzed for feedback content, delivery mechanisms, barriers to feedback, and outcomes of feedback using archival data. Pattern analysis was conducted to examine relations between A&F components and changes in MBC use. RESULTS: Several sites utilized both aggregate and individualized data summaries, and data accuracy concerns were common. Feedback cycles featuring individual-level clinician data, data relevant to MBC barriers, and information requested by data recipients were related to patterns of increased MBC use. CONCLUSIONS: These findings support extant theory, such as Feedback Intervention Theory. Mental health professionals wishing to apply A&F should consider establishing reciprocal feedback mechanisms on the quality and amount of data being received and adopting specific roles communicating and addressing data quality concerns. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT02266134.

Taking a Magnifying Glass to Measurement-Based Care Consultation Sessions: with What Issues Do Mental Health Clinicians Struggle?

PURPOSE: Ongoing consultation following initial training is one of the most commonly deployed implementation strategies to facilitate uptake of evidence-based practices, such as measurement-based care (MBC). Group consultation provides an interactive experience with an expert and colleagues to get feedback on actual issues faced, yet there is little research that unpacks the questions raised in consultation and what types of issues are important to address. METHODS: The current study characterized the questions and concerns raised by community mental health clinicians (N = 38 across six clinics) during group consultation sessions completed as part of an MBC implementation trial. We conducted a qualitative content analysis of consultation forms completed by clinicians before each MBC consultation session. RESULTS: Clinicians sought MBC consultation for clients across a range of ages and levels of depression severity. Qualitative results revealed five main questions and concerns in consultation sessions: (1) how to administer the PHQ-9, (2) how to review PHQ-9 scores, (3) how to respond to PHQ-9 score, (4) the types of clients for whom MBC would be appropriate, and (5) how MBC could impact a clinician's usual care. CONCLUSION: Findings highlight the need for ongoing consultation and limitations of workshop training alone. Practical recommendations for addressing the common questions and concerns identified are presented to support MBC use.

Using a most significant change approach to evaluate learner-centric outcomes of clinical scholars leadership training program.

BACKGROUND: Most significant change (MSC) analysis is an evaluative method used to supplement outcome and impact program evaluations. MSC stories provide mini-narratives contextualizing the effect a program had on an individual. The Clinical Scholars program evaluation team used MSC to explore learner-centric leadership outcomes of the Clinical Scholars leadership training program. PURPOSE OF THE RESEARCH: To identify thematic outcomes of trainees during the three years of their leadership training in the Clinical Scholars program. RESULTS: We identified three central thematic areas of most significant change: (1) leadership in practice, (2) self-awareness, and (3) equity, diversity, and inclusion. Other present themes included stakeholder engagement, effective leadership, collaboration, impact, expanded influence, innovation, networking, community engagement, evidence-based, implementation science, organizational development, hope, and advocating within organizations, teams, and the community. A total of 25 mid- and advanced-career level participants represented a diverse array of disciplines within the healthcare sector. CONCLUSIONS: MSC stories represented leaders' individual and team growth due to participating in the Clinical Scholars program. Results confirmed fidelity of program implementation to the original program goals, as outlined in the grant proposal. They highlighted three critical areas of development for this cohort of Fellows.

Standardized Versus Tailored Implementation of Measurement-Based Care for Depression in Community Mental Health Clinics.

Objective: Measurement-based care (MBC) is an evidence-based practice that is rarely integrated into psychotherapy. The authors sought to determine whether tailored MBC implementation can improve clinician fidelity and depression outcomes compared with standardized implementation. Methods: This cluster-randomized trial enrolled 12 community behavioral health clinics to receive 5 months of implementation support. Clinics randomized to the standardized implementation received electronic health record data captured with the nine-item Patient Health Questionnaire (PHQ-9), a needs assessment, clinical training, guidelines, and group consultation in MBC fidelity. Tailored implementation support included these strategies, but the training content was tailored to clinics' barriers to MBC, and group consultation centered on overcoming these barriers. Clinicians (N=83, tailored; N=71, standardized) delivering individual psychotherapy to 4,025 adults participated. Adult patients (N=87, tailored; N=141, standardized) contributed data for depression outcome analyses. Results: The odds of PHQ-9 completion were lower in the tailored group at baseline (odds ratio [OR]=0.28, 95% CI=0.08­0.96) but greater at 5 months (OR=3.39, 95% CI=1.00­11.48). The two implementation groups did not differ in full MBC fidelity. PHQ-9 scores decreased significantly from baseline (mean±SD=17.6±4.4) to 12 weeks (mean=12.6±5.9) (p<0.001), but neither implementation group nor MBC fidelity significantly predicted PHQ-9 scores at week 12. Conclusions: Tailored MBC implementation outperformed standardized implementation with respect to PHQ-9 completion, but discussion of PHQ-9 scores in clinician-patient sessions remained suboptimal. MBC fidelity did not predict week-12 depression severity. MBC can critically inform collaborative adjustments to session or treatment plans, but more strategic system-level implementation support or longer implementation periods may be needed.

Training "Pivots" from the Pandemic: Lessons Learned Transitioning from In-Person to Virtual Synchronous Training in the Clinical Scholars Leadership Program.

INTRODUCTION: Since the inception of distance-based teaching modalities, a debate has ensued over the quality of online versus in-person instruction. Due to the COVID-19 pandemic, a number of teaching environments-including leadership development trainings for post-graduate learners-have been thrust into exploring the virtual learning environment more thoroughly. One three-year leadership development program for interdisciplinary healthcare professionals transitioned three simultaneous leadership intensives from in-person to online in the spring of 2020. METHODS: Documented changes in overall training length, session length, and session format are described. Further, evaluative data were collected from participants at both retreats via post-session surveys. Ninety-three participants attended the 2019 retreat, and 92 participants attended the 2020 virtual retreat. Quantitative data of three rating questions per session are reported: 1) overall session satisfaction, 2) participants' reported knowledge gain, and 3) participants' reported ability gain. Qualitative data were obtained via two open-ended feedback questions per session. RESULTS: In comparing pre/post scores for knowledge and ability, participants had meaningful (and in some cases higher) self-reported gains in knowledge and ability measures in the online environment, as compared to the in-person environment. Participants reported statistically significant gains in all sessions for both knowledge and ability. Qualitative data of participant feedback identified a number of positive themes similar across the in-person and virtual settings. Negative or constructive feedback of the virtual setting included time constraint issues (eg too much content in one session, a desire for more sessions overall), technical difficulties, and the loss of social connection and networking with fellow participants as compared to in-person trainings. DISCUSSION: While meaningful shifts in knowledge and ability ratings indicate that the transition to successful online learning is possible, several disadvantages remain. The preparation time for both faculty and participants was considerable, there is a need to reduce overall content in each session due to time restraints, and participants indicated feeling the loss of one-on-one connections with their peers in the training. Lessons learned of transitioning leadership training from in-person to an online experience are highlighted.

In Search of Reliability: Expert-Informed Training Methods for Conducting Observational Coding of Cognitive Behavioral Therapy Fidelity.

BACKGROUND: Observational coding is the gold standard for measuring treatment fidelity; however, the intensive training needed for reliable and valid measurement has not been carefully scrutinized. A systematic review concluded there is a lack of information in published studies on how to approach training raters, and the available content suggests widely variable approaches are taken across research teams. The quality and comparability of the data produced from these treatment fidelity measures is undermined by heterogeneous training approaches occurring both within and between coding teams. No guidance for training teams to do observational coding for fidelity is currently available. The aims of the current study were to: (a) characterize expert-informed processes for observational coding regarding training, coding, and achieving reliability, and (b) generate expert recommendations for training and conducting observational coding. METHOD: A semi-structured interview was used to explore international CBT expert's (N = 11) perspectives and experiences regarding observational coding training processes. A qualitative content analysis approach was used to analyze the data. RESULTS: Experts in the study provided information about their training, coding, and reliability processes. CONCLUSION: Results from this study informed recommendations on how to: (a) introduce scale items, (b) anchor ratings, and (c) drift prevention.

Association between perceived risk of harm and self-reported binge drinking, cigarette smoking, and marijuana smoking in young adults.

OBJECTIVES: Evaluate the association between perceived risk of harm and self-reported binge drinking, cigarette smoking, and marijuana smoking among college students. PARTICIPANTS: Participants were 599 students (ages 19-28) at a large Midwestern university recruited from October 2015 to December 2017. METHODS: Hurdle regression was used to test the relationship between perceived risk of harm from substance use (i.e., binge drinking, cigarette smoking, and marijuana smoking), and self-reported use. Demographic characteristics were tested as moderators of this relationship. RESULTS: Engagement in all three substance use behaviors was less likely when perceived risk was high. Age moderated the association between perceived risk and self-reported marijuana smoking with younger participants demonstrating a stronger relationship between perceived risk of smoking marijuana and self-reported marijuana smoking. CONCLUSION: Intervention programs will be most effective when perceived risk of substance use is high. Therefore, intervention programs should aim to increase college students' perceived risk of substance use.